System and method for enabling collaborative media stream editing

ABSTRACT

A system and method for editing multimedia content. A server transmits multimedia content to a plurality of electronic devices. Each of the plurality of electronic devices enables a user to perform at least one action in relation to the multimedia content. A a voting engine gathers information concerning the actions performed by users in relation to the multimedia content transmitted to the electronic devices. In response to the information gathered by the voting engine, the multimedia content is modified based upon predefined criteria. The modified multimedia content is subsequently provided to later users.

FIELD OF THE INVENTION

The present invention relates generally to the editing of multimediacontent. More particularly, the present invention relates to thecollaborative editing of multimedia content by a plurality of usersbased upon actions of the users.

BACKGROUND OF THE INVENTION

Video images are extremely attractive and effective in conveyinginformation to people. This is due in large part to the exceptionalcapability of human eyes to process a viewed image. A person can view avery brief video clip, yet still obtain and retain important informationfrom the clip. This is clearly demonstrated by the prevalence oftelevision commercials and music videos comprising very short “cuts” orvideo clips.

In recent years, devices for recording video streams have becomeincreasingly commonplace. In particular, camera phones and digitalcameras have exploded in popularity, creating an enormous multimediarecorder base. Additionally, Internet-connected personal computers andother communication-capable devices make interactive media streamconsumption possible, and the storage capability in these devices hasbecome very inexpensive.

At the same time that the ability to record video streams has progressedamong users, “collaborative editing” has also become more commonplace.In collaborative editing, multiple users are capable of working togetherin order to edit a video stream as necessary or desired. For example,multiple users can collaborate through different devices to cut,combine, or otherwise modify video streams for future viewing. However,even though these types of edits can be made to video streams bymultiple users simultaneously, these edits are of the “active” variety,i.e., through the use of multimedia editing programs or similarsoftware, where the purpose of the activity is to edit the stream. Othersystems comprise “unorganized” collaborative editing, where theindividual users are not aware of others who may also be influencing theediting process. In such systems, the resulting content may be theresult of a “behaviour analysis” process, wherein the actions of theindividual users is used as an input.

SUMMARY OF THE INVENTION

The present invention provides for a system of collaborative editing ofmedia streams based upon actions of earlier viewers. The presentinvention enables collaborative editing and consumption of media streamsof raw material produced by the greater public or members of acommunity. The raw media stream is edited or refined according to theactions of those individuals who view the material, resulting in a“passive” editing process. The viewer actions are generated with acontroller device, such as a mobile phone, that uses a controlconnection to a media processing server or separate voting engine thatmay be attached or integrated to the playback device.

Collaborative editing according to the present invention can result inthe creation of entirely new compositions. The collaborative editing ofthe present invention may be either explicit or implicit and may affecteither the entire media stream or only a portion of the stream. In thepresent invention, a media element or component can be presented in anew context, as a part of a composition where the element has not beenpreviously included, or has been included in some other manner. In thepresent invention, a new video clip or picture can be used to present anevent or place which was previously presented in the composition. Mediaelements may also be combined by, for example, mixing a new soundtrackwith a video or picture.

The present invention may also involve the manipulation of audioinformation in addition to video information. For example, implicitcollaboration may be used to adjust the volume of a selected audioportion of a media stream. Additionally, if users choose their own musicto be played with a video clip, the music selections may be used toselect a soundtrack for future users.

The present invention provides a significant improvement over currentediting systems. With collaborative editing, the creation of highlyusable media streams can be accomplished with shared efforts by viewerswithout the need for paid or professional editing work. Multimediamessaging also becomes more user friendly with the present invention bybetter exploiting human capabilities in playback situations.Collaborative editing according to the present invention can occureither explicitly or implicitly, and the editing can occur continuously,resulting in a continuous refinement of the media stream at issue.

These and other objects, advantages and features of the invention,together with the organization and manner of operation thereof, willbecome apparent from the following detailed description when taken inconjunction with the accompanying drawings, wherein like elements havelike numerals throughout the several drawings described below.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an overview diagram of a system within which the presentinvention may be implemented;

FIG. 2 is a perspective view of a mobile telephone that can be used inthe implementation of the present invention;

FIG. 3 is a schematic representation of the telephone circuitry of themobile telephone of FIG. 2;

FIG. 4 is a representation of a system including a recording device, amedia processing server, and a playback device according to oneembodiment of the present invention; and

FIG. 5 is a flow chart showing the implementation of one embodiment ofthe present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The present invention is based on the automated composition of a rawmedia stream from multi media or plain still image material recorded byindividuals or observation cameras, such as web cams. A typicalcomposition method according to the present invention involvescollecting media items offered in a category selected by the recordingindividual in a sequence according to recording time.

The raw media stream is edited or refined according to actions of thoseindividuals who view the material. A typical editing method according tothe present invention involves a system that obtains informationconcerning the viewing actions of individual viewers. When an image orstream is displayed that is of a particular interest to an individual,the individual users can browse the multimedia stream in many ways. Theviewing actions that are involved in the editing process can comprisetypical implicit editing methods, and the outcome is based upon ananalysis of the actions by those individuals who view the material.Actions that can be used in the collaborative editing process include,but are not limited to, the use of an “accelerated playback” mode,viewing the stream at a normal playback speed, repeatedly viewing thesame portion of content, adjusting the volume, bass, balance, fade,treble or other audio-related aspects of the content, zooming within themultimedia content, panning within the multimedia content, or viewingthe content in slow motion. As used herein, “panning” refers to, if apicture ratio in a video is wider than a device's display, adjusting thepositioning of the video so that a selected portion of the video remainsviewable by the user. All of the above actions can be considered asindications of interest to that portion of the media stream. Informationconcerning all of these actions can be consolidated with a conventionaldata mining technique or similar method in order to refine the appearsappearance of the media stream for subsequent users and/or viewers.

FIG. 4 is a representation showing a collaborative media stream editingprocess according to one embodiment of the present invention. As shownin FIG. 4, a recording device 100 is used to capture an image or a videostream, referred to herein generically as content 110, and transmit thecontent 110 to a processing server 120. The recording device 100 cantake a variety of forms, including, but not limited to, a digital movierecorder, a conventional movie recorder, a digital camera, and a mobiletelephone with built-in camera capabilities.

The processing server 120 transmits the content 110 to at least oneplayback device 130. The playback device 130 can also take a widevariety of forms, including virtually any device that includes some formof display unit. The playback device 130 can include either a built-inor an otherwise associated controller 140 that enables the viewer tomanipulate the media stream. The controller 140 can include items suchas a joystick, keyboard keys, touch pads, a touch-sensitive display, ora variety of other features that allow the viewer to manipulate thecontent 110. In one embodiment of the invention, a mobile telephoneserves as the controller 140.

The system depicted in FIG. 4 also includes a voting engine 150. Thevoting engine 150 is used to collect information concerning theactivities of each individual viewer. The voting engine 150 keeps trackof how many viewers perform particular actions around individual contentsegments, including whether viewers fast forward past a segment, whethera viewer watches a segment multiple times, etc. The voting engine 150 isincorporated into the processing server 120 in one embodiment of theinvention. The voting engine 150 can also be a stand-alone device. Inaddition, the voting engine 150 can also be incorporated into therecording device 100, into one or more playback devices 130, or into oneor more controllers 140.

In one embodiment of the invention, several individuals simultaneouslyview the media stream in a broadcast mode using respective playbackdevices 130. The playback devices 130 can each include their owncontroller 140 or other mechanism, or a single playback device 130 canbe connected to multiple controllers 140, where individuals can view andcontrol the content 120 through one of the controllers. Each user orviewer can use a controller 140 to generate viewer actions. The vieweractions are processed by the voting engine 150. This information can beused by the voting engine 150 to generate an altered or editedappearance of the media stream and to control the playback of the mediastream.

Editing actions that can be implemented by a playback device 130 inorder to alter the appearance of the content include, but are notlimited to: adjusting the time to keep a still image on the display,selecting the part of a video clip to be exhibited at a slower speed,selecting a still image or frame to be shown from a video clip whenbrowsing a video stream, adjusting the volume, bass, treble, balance,fade or other audio-related aspects of a content portion, zooming withinthe multimedia content, panning within the multimedia content, andskipping a portion of a video stream.

Editing actions, such as those discussed above, result in the generationof a modified appearance for the media stream for future viewing. Forexample, in the event that several viewers “rewind” a portion of a videostream in order to watch the portion more than one time, it may bedetermined that that particular portion should be given more prominencewhen exhibited to future users. As a result, the media processing server120 may adjust the media stream for future users so that the particularportion is shown in slow motion. Conversely, portions which are skippedover by users could be deemphasized for future viewers. Otherpossibilities include the showing of a more detailed view of the contentportion, an accelerated rate of exhibiting the piece of content, orother actions. In any event, the end result of this collaborativeediting process is the subsequent display of the content in a modifiedfashion that is likely to be of more interest to the user than thecontent in an unmodified form.

It should be noted that the media stream can include not only videoand/or still images but also other media forms such as audio. In suchsituations, the different portions can edited independent from eachother in one embodiment of the invention. For example, an audio portionof a media stream can be independently edited without editing the videoor image portion of the content.

In one particular implementation of the present invention, a basicmobile telephone with a built-in camera is used as a recording device100, which is used to post content 110 to a selected raw media channel.The media processing server 120 receives posts to the raw media channeland creates online and browsing media streams according to the actionsof the individual viewers. A playback device, such as a multimediaplayback-capable telephone or personal computer, is able to receive andplay the stream. According to this embodiment of the invention, a numberof the playback devices need to have the ability to obtain informationregarding the viewer actions and to communicate information about theactions to the media processing server 120. Some playback devices may beable to react to viewer actions locally while, at the same time,consuming a media stream received from the media processing server 120.

FIG. 5 is a flow chart showing the implementation of one embodiment ofthe present invention. At 200, a recording device 100 records content110 from a source. In a particular embodiment of the invention, this canbe in the form of live-action recording of video, although it is alsopossible that the content 110 can take the form of a series of stillimages, or the content 110 can take other forms. At step 210, thecontent 110 is transmitted to the media processing server 120. It shouldbe noted that the content 110 can be transmitted in real time to themedia processing server 120, or the content 110 can be stored in therecording device 110 and transmitted at a later time to the mediaprocessing server. At step 220, the content 110 is transmitted to atleast one playback device 130. At step 230, the content 110 is viewed byat least one user. At step 240, each user uses his or her controller 140to view the content 110 in a specific way, e.g., by fast-forwardingthrough portions, playing portions in slow motion, etc. At step 250,information concerning how the content 110 is viewed is transmitted tothe voting engine 150, which collects this information from each userthat views the content 110. At step 260, the voting engine uses theinformation to generate modified content, which is transmitted tosubsequent viewers at step 270.

The generation of modified content can occur according to predefinedcriteria. “Predefined criteria” can be both relative and absolute andcan be based upon the number of viewers of content, the number ofconsecutive viewers performing the same action, the frequency ofperforming a particular action by viewers or a wide variety of othersuitable measures. For example, the predefined criteria can state that,if more than a certain percentage of viewers fast-forward through aparticular segment, then the modified content should exhibit thatportion of the content at an accelerated rate in the future. The content110 then continues to be viewed and modified over time. The generationof modified content can occur through the use of an engine having anartificial intelligence. In this situation, the objective of theartificial intelligence is to generate content which would match theneeds and tastes of an average consumer, or content which would matchthe needs of a “key user” of a specific target group.

According to one embodiment of the invention, a particular playbackdevice 130 may access the content 110 with an indication to view thecontent 110 only when it has been altered and edited such that thecontent has obtained a desired level of “maturity” quality.

FIG. 1 shows a system 10 in which the present invention can be utilized,comprising multiple communication devices that can communicate through anetwork. The system 10 may comprise any combination of wired or wirelessnetworks including, but not limited to, a mobile telephone network, awireless Local Area Network (LAN), a Bluetooth personal area network, anEthernet LAN, a token ring LAN, a wide area network, the Internet, etc.The system 10 may include both wired and wireless communication devices.

For exemplification, the system 10 shown in FIG. 1 includes a mobiletelephone network 11 and the Internet 28. Connectivity to the Internet28 may include, but is not limited to, long range wireless connections,short range wireless connections, and various wired connectionsincluding, but not limited to, telephone lines, cable lines, powerlines, and the like.

The exemplary communication devices of the system 10 may include, butare not limited to, a mobile telephone 12, a combination PDA and mobiletelephone 14, a PDA 16, an integrated messaging device (IMD) 18, adesktop computer 20, and a notebook computer 22. The communicationdevices may be stationary or mobile as when carried by an individual whois moving. The communication devices may also be located in a mode oftransportation including, but not limited to, an automobile, a truck, ataxi, a bus, a boat, an airplane, a bicycle, a motorcycle, etc. Some orall of the communication devices may send and receive calls and messagesand communicate with service providers through a wireless connection 25to a base station 24. The base station 24 may be connected to a networkserver 26 that allows communication between the mobile telephone network11 and the Internet 28. The system 10 may include additionalcommunication devices and communication devices of different types.

The communication devices may communicate using various transmissiontechnologies including, but not limited to, Code Division MultipleAccess (CDMA), Global System for Mobile Communications (GSM), UniversalMobile Telecommunications System (UMTS), Time Division Multiple Access(TDMA), Frequency Division Multiple Access (FDMA), Transmission ControlProtocol/Internet Protocol (TCP/IP), Short Messaging Service (SMS),Multimedia Messaging Service (MMS), e-mail, Instant Messaging Service(IMS), Bluetooth, IEEE 802.11, etc. A communication device maycommunicate using various media including, but not limited to, radio,infrared, laser, cable connection, and the like.

FIGS. 2 and 3 show one representative mobile telephone 12 within whichthe present invention may be implemented. It should be understood,however, that the present invention is not intended to be limited to oneparticular type of mobile telephone 12 or other electronic device. Themobile telephone 12 of FIGS. 2 and 3 includes a housing 30, a display 32in the form of a liquid crystal display, a keypad 34, a microphone 36,an ear-piece 38, a battery 40, an infrared port 42, an antenna 44, asmart card 46 in the form of a UICC according to one embodiment of theinvention, a card reader 48, radio interface circuitry 52, codeccircuitry 54, a controller 56 and a memory 58. Individual circuits andelements are all of a type well known in the art, for example in theNokia range of mobile telephones.

The present invention is described in the general context of methodsteps, which may be implemented in one embodiment by a program productincluding computer-executable instructions, such as program code,executed by computers in networked environments.

Generally, program modules include routines, programs, objects,components, data structures, etc. that perform particular tasks orimplement particular abstract data types. Computer-executableinstructions, associated data structures, and program modules representexamples of program code for executing steps of the methods disclosedherein. The particular sequence of such executable instructions orassociated data structures represents examples of corresponding acts forimplementing the functions described in such steps.

Software and web implementations of the present invention could beaccomplished with standard programming techniques with rule based logicand other logic to accomplish the various database searching steps,correlation steps, comparison steps and decision steps. It should alsobe noted that the words “component” and “module” as used herein, and inthe claims, is intended to encompass implementations using one or morelines of software code, and/or hardware implementations, and/orequipment for receiving manual inputs.

The foregoing description of embodiments of the present invention havebeen presented for purposes of illustration and description. It is notintended to be exhaustive or to limit the present invention to theprecise form disclosed, and modifications and variations are possible inlight of the above teachings or may be acquired from practice of thepresent invention. The embodiments were chosen and described in order toexplain the principles of the present invention and its practicalapplication to enable one skilled in the art to utilize the presentinvention in various embodiments and with various modifications as aresuited to the particular use contemplated.

1. A method of editing multimedia content, comprising: providingmultimedia content to at least one viewer; observing actions performedby the at least one viewer in relation to the provided multimediacontent; and in response to the observed actions, modifying themultimedia content based upon predefined criteria; and providing themodified multimedia content to at least one subsequent viewer.
 2. Themethod of claim 1, further comprising, before providing the multimediacontent to the at least one viewer, receiving the multimedia contentfrom a recording device.
 3. The method of claim 1, wherein the actionsare selected from the group consisting of fast forwarding through aportion of the multimedia content, replaying a portion of the multimediacontent, viewing a portion of the multimedia content at aslower-than-normal rate; skipping a portion of the multimedia content,manipulating audio-related aspects of the multimedia content, zoomingwithin the multimedia content, panning within the multimedia content andcombinations thereof.
 4. The method of claim 1, wherein the multimediacontent is modified in a manner selected from the group consisting ofeliminating a portion of the multimedia content, modifying a portion ofthe multimedia content to be played at a slower-than-normal rate,modifying a portion of the multimedia content to be played at afaster-than-normal rate, zooming into an image presented in themultimedia content, panning within the multimedia content, modifyingaudio-related aspects of the multimedia content and combinationsthereof.
 5. The method of claim 1, wherein the at least one viewer usesa controller to perform the observed actions.
 6. The method of claim 1,wherein the predefined criteria comprise instructions indicating that,if more than a certain percentage or number of viewers performs anaction on a portion of the multimedia content, modifying the multimediacontent in a manner substantially identical to the performed action. 7.A computer program product for editing multimedia content, comprising:computer code for providing multimedia content to at least one viewer;computer code for observing actions performed by viewers in relation tothe provided multimedia content; and computer code for in response tothe observed actions, modifying the multimedia content based uponpredefined criteria; and computer code for providing the modifiedmultimedia content to at least one subsequent viewer.
 8. The computerprogram product of claim 7, further comprising computer code for, beforeproviding the multimedia content to the at least one viewer, receivingthe multimedia content from a recording device.
 9. The computer programproduct of claim 7, wherein the actions are selected from the groupconsisting of fast forwarding through a portion of the multimediacontent, replaying a portion of the multimedia content, viewing aportion of the multimedia content at a slower-than-normal rate; skippinga portion of the multimedia content, manipulating audio-related aspectsof the multimedia content, zooming within the multimedia content,panning within the multimedia content and combinations thereof.
 10. Thecomputer program product of claim 7, wherein the multimedia content ismodified in a manner selected from the group consisting of eliminating aportion of the multimedia content, modifying a portion of the multimediacontent to be played at a slower-than-normal rate, modifying a portionof the multimedia content to be played at a faster-than-normal rate,panning within the multimedia content, zooming into an image presentedin the multimedia content, modifying audio-related aspects of themultimedia content and combinations thereof.
 11. The computer programproduct of claim 7, wherein the predefined criteria compriseinstructions indicating that, if more than a certain percentage ornumber of viewers performs an action on a portion of the multimediacontent, modifying the multimedia content in a manner substantiallyidentical to the performed action.
 12. A system for editing multimediacontent, comprising: a server for transmitting multimedia content; aplurality of electronic devices in communication with the server forreceiving the multimedia content, each of the plurality of electronicdevices enabling a user to perform at least one action in relation tothe multimedia content; and a voting engine configured to gatherinformation concerning the actions performed by users in relation to themultimedia content transmitted to the plurality of electronic devices,wherein, in response to the information gathered by the voting engine,.the server modifies the multimedia content based upon predefinedcriteria, and wherein the modified multimedia content is subsequentlyprovided to the plurality of electronic devices.
 13. The system of claim12, wherein the at least one action is selected from the groupconsisting of fast forwarding through a portion of the multimediacontent, replaying a portion of the multimedia content, viewing aportion of the multimedia content at a slower-than-normal rate; skippinga portion of the multimedia content, manipulating audio-related aspectsof the multimedia content, zooming within the multimedia content,panning within the multimedia content and combinations thereof.
 14. Thesystem of claim 12, wherein the multimedia content is modified in amanner selected from the group consisting of eliminating a portion ofthe multimedia content, modifying a portion of the multimedia content tobe played at a slower-than-normal rate, modifying a portion of themultimedia content to be played at a faster-than-normal rate, zoominginto an image presented in the multimedia content, panning within themultimedia content, modifying audio-related aspects of the multimediacontent and combinations thereof.
 15. The system of claim 12, whereinthe voting engine is located within the server.
 16. The system of claim12, further comprising a recording device, the recording deviceproviding the multimedia content to the server before the multimediacontent is transmitted to the plurality of electronic devices.
 17. Adevice, comprising: a processor; and a memory unit operatively connectedto the processor and including: computer code for receiving multimediacontent from a remote location; computer code for providing themultimedia content to a viewer; computer code for observing actionsperformed by the viewer in relation to the multimedia content; computercode for transmitting information concerning the observed actions to theremote location; computer code for receiving the multimedia content inmodified form from the remote location.
 18. The device of claim 17,wherein the actions are selected from the group consisting of fastforwarding through a portion of the multimedia content, replaying aportion of the multimedia content, viewing a portion of the multimediacontent at a slower-than-normal rate; skipping a portion of themultimedia content, manipulating audio-related aspects of the multimediacontent, zooming within the multimedia content, panning within themultimedia content and combinations thereof.
 19. The device of claim 17,further comprising a controller operatively connected to the processor,the controller permitting the user to perform the observed actions. 20.A network element for editing multimedia content, comprising: aprocessor; and a memory unit operatively connected to the processor andincluding: computer code for receiving information concerning actionsthat have been observed being performed on multimedia content; computercode for, in response to the observed actions, modifying the multimediacontent based upon predefined criteria; and computer code fortransmitting the modified multimedia content to a remote location forsubsequent exhibition to a viewer.
 21. The network element of claim 20,wherein the memory unit includes a voting engine configured to gatherinformation concerning the observed actions in relation to themultimedia content.
 22. The network element of claim 20, wherein themultimedia content is modified in a manner selected from the groupconsisting of eliminating a portion of the multimedia content, modifyinga portion of the multimedia content to be played at a slower-than-normalrate, modifying a portion of the multimedia content to be played at afaster-than-normal rate, zooming into an image presented in themultimedia content, panning within the multimedia content, modifyingaudio-related aspects of the multimedia content and combinationsthereof.